Skip to content

[AIT-685] Fix View re-rendering by caching flattenNodes and skipping tree walks during streaming#40

Merged
VeskeR merged 3 commits intomainfrom
AIT-685/fix-react-rerendering
Apr 16, 2026
Merged

[AIT-685] Fix View re-rendering by caching flattenNodes and skipping tree walks during streaming#40
VeskeR merged 3 commits intomainfrom
AIT-685/fix-react-rerendering

Conversation

@VeskeR
Copy link
Copy Markdown
Contributor

@VeskeR VeskeR commented Apr 13, 2026

Summary

  • Cache flattenNodes() in the View so React hooks get O(1) lookups
    instead of re-walking the entire tree on every streaming token
  • Add a structuralVersion counter to the Tree so the View can
    distinguish content-only updates (streaming) from structural changes
    (new nodes, deletions, branch reorders) and skip the tree walk entirely
  • Preserve stable object references for unchanged messages so
    React.memo can skip re-rendering them

Resolves AIT-685

@VeskeR VeskeR changed the title [AIT-685] Fix react re-rendering everything because all objects are rebuilt [AIT-685] Fix View re-rendering by caching flattenNodes and skipping tree walks during streaming Apr 13, 2026
@VeskeR VeskeR force-pushed the AIT-685/fix-react-rerendering branch from 3a92db6 to 4cbcb92 Compare April 13, 2026 13:29
@VeskeR VeskeR requested a review from ttypic April 13, 2026 13:32
@VeskeR VeskeR force-pushed the AIT-685/fix-react-rerendering branch from 4cbcb92 to 3e7e049 Compare April 13, 2026 13:49
@github-actions github-actions bot temporarily deployed to staging/pull/40/typedoc April 13, 2026 13:50 Inactive
@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 13, 2026

Coverage Report

Status Category Percentage Covered / Total
🔵 Lines 92.66% 2109 / 2276
🔵 Statements 90.96% 2255 / 2479
🔵 Functions 93.25% 387 / 415
🔵 Branches 77.85% 949 / 1219
File Coverage
File Stmts Branches Functions Lines Uncovered Lines
Changed Files
src/core/transport/tree.ts 96.46% 88.63% 100% 100% 150, 175, 239, 283, 305, 307, 347
src/core/transport/view.ts 86.52% 72.34% 98.11% 89.93% 241, 245-246, 253-254, 259-260, 278, 294, 356, 428, 440-443, 459-467, 516, 519, 522, 545, 556, 567, 596-599, 690, 751-762
Generated in workflow #146 for commit 324ff7f by the Vitest Coverage Report Action

Copy link
Copy Markdown
Contributor

@ttypic ttypic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@VeskeR VeskeR force-pushed the AIT-685/fix-react-rerendering branch from 3e7e049 to 7de19dd Compare April 14, 2026 15:39
@github-actions github-actions bot temporarily deployed to staging/pull/40/typedoc April 14, 2026 15:40 Inactive
VeskeR added 3 commits April 16, 2026 13:36
Regression tests for incoming changes for AIT-685.

Guard the contract that unchanged messages keep stable object
references during streaming updates. These tests pass today and
protect against regressions as we optimize the View's update path:

- View: unchanged message refs preserved after content-only upsert
- View: array ref changes after update (so React detects it)
- View: simulated streaming — only the active message ref changes
- useView: message refs stable across mock streaming updates
- useMessageSync: message refs stable across mock streaming updates
During token streaming, every tree update caused multiple O(n) tree
traversals: once inside the View's _onTreeUpdate(), then again for
each React hook subscriber calling view.flattenNodes(). With rapid
streaming tokens this multiplied into a significant performance cost.

The View now caches the flattenNodes() result in a _cachedNodes field.
The public flattenNodes() returns this cache in O(1). A new private
_computeFlatNodes() method performs the actual tree walk and is called
only when the visible output may have changed: tree structural changes,
branch selection changes, fork auto-selection, and history page reveal.

The original design avoided caching ("no cache invalidation complexity,
at the cost of repeated traversals") because the result depends on
branch selection state. Caching is safe here because every mutation
path that can change the visible output refreshes the cache
synchronously before emitting events to consumers - JS single-threading
eliminates async staleness risks.
During token streaming, every tree update triggered a full O(n) tree
walk even though the tree structure hadn't changed - only a single
message's content was updated. With conversations of hundreds of
messages and tokens arriving at high frequency, this became the
dominant cost on the hot path.

The Tree now exposes a structuralVersion counter (on TreeInternal)
that increments on insert, delete, and serial promotion - but not
on content-only upsert of an existing node. The View compares this
counter against its last-seen version: when unchanged, it takes a
fast path that compares cached message references against the
snapshot in O(visible_count) instead of re-walking the tree.

A monotonic counter was chosen over alternatives (tagged events,
dirty sets, boolean flags) because it requires no public API change,
is multi-View safe (each View tracks its own last-seen version),
and cannot produce false negatives.
@VeskeR VeskeR force-pushed the AIT-685/fix-react-rerendering branch from 7de19dd to 324ff7f Compare April 16, 2026 12:37
@VeskeR VeskeR merged commit 1b4ccbd into main Apr 16, 2026
13 checks passed
@VeskeR VeskeR deleted the AIT-685/fix-react-rerendering branch April 16, 2026 12:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants